Skip to content

Conversation

@Shahroz16
Copy link
Contributor

@Shahroz16 Shahroz16 commented Mar 25, 2025

Problem

Our previous testing strategy relied on file snapshots, which became brittle when testing across multiple Expo versions as these generate slightly different native projects.

Previous Approach

  • Full File Snapshots: Entire files were saved as snapshots and compared exactly
  • Inline Snapshots: Used for files that needed dynamic app name injection
  • Brittle Comparisons: Any change in Expo's generated code would break tests
  • All-or-Nothing: Tests passed only if generated files were identical to snapshots

Current Approach

  • Structured Data Testing: Parse files into structured objects (JSON, XML) and assert on important parts
  • Targeted Assertions: Check specific elements that should be present regardless of the Expo version
  • Pattern Matching: Use contains/matching rather than exact equality
  • Contract Testing: Define what plugin "promises" and verify those contracts are fulfilled

Key Improvements

  1. Flexibility vs Precision:
    - Before: High precision but brittle (exact file matching)
    - Now: Flexible but still thorough (structural matching)
  2. Test Focus:
    - Before: "Does the output match exactly what we expect?"
    - Now: "Does the output contain the elements required for functionality?"
  3. Maintenance Burden:
    - Before: Update all snapshots when Expo versions change
    - Now: Tests remain valid across Expo versions unless core functionality changes
  4. Failure Information:
    - Before: Failed tests only showed file differences
    - Now: Failed tests show exactly which functional requirements weren't met
  5. Separation of Concerns
    - Test only what matters for each integration point

Solution:

Implemented a comprehensive testing strategy focusing on functionality rather than exact implementation:

  1. Contract Testing: Define and verify what our plugin promises to deliver regardless of Expo version
  2. Structured Data Testing: Parse files (Gradle, XML, etc.) into objects and assert only on critical properties
  3. Flexible Pattern Matching: Test for the presence of required code patterns instead of exact equality
  4. Custom Jest Matchers: Create readable assertions that encapsulate complex validation logic
  5. Parameterized Tests: Support running tests with different configurations

@Shahroz16 Shahroz16 self-assigned this Mar 25, 2025
Base automatically changed from unit-test-multi-expo-versions to beta March 25, 2025 09:10
@Shahroz16 Shahroz16 marked this pull request as draft April 28, 2025 10:43
@Shahroz16
Copy link
Contributor Author

putting these back to draft, so when we plan to bring expo improvements project again, we can restart the conversation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants